Using Optical Flow as an Additional Constraint for Solving the Correspondence Problem in Binocular Stereopsis

A stereo matching algorithm for 3D structure reconstruction relies on the correlation of image features in a pair of images. Usually, two calibrated cameras are used to capture scenes, and the left and right images are rectified to reduce the search space from 2D image region to 1D line. Then, the disparity is defined by the horizontal difference between the corresponding points on the search line. To solve the correspondence problem, various image features such as image intensity, edge, color, infrared light, pixel motion, etc. are employed either separately or integrated together. Among these image features, pixel motion or optical flow has been rarely used as one of the matching features (A. Scheuing & H. Niemann, 1986, G. Sudhir et al, 1995, A. M. Waxman & J. H. Duncan, 1986). In this chapter, we focus on the use of optical flow as an additional constraint in solving the correspondence problem. The stereo matching algorithm for 3D structure reconstruction can be divided into two groups based on the matching primitives: intensity based matching (also referred to as area based matching) and feature based matching. The intensity based matching method searches the best matching points using only intensity values of pixels. The method can further be divided into two groups depending on the smoothness constraints in minimization of matching cost function: local (window-based) method and global method. More details on the intensity based method can be found in a survey by Scharstein (D. Scharstein & R. Szeliski. 2002, M. Z. Brown et al, 2003). The intensity based matching method produces a dense disparity map without any additional post-processing, but usually needs an exhaustive search. Since the intensity based method uses the intensity value at each pixel directly, this method may suffer from the varying illumination problem. The feature based matching method searches the best matching points using some special symbolic feature points, such as line, contour, corner, etc. Since this method calculates disparity values only on the pixels corresponding to the feature point, this method does not require an exhaustive search. Also, symbolic features are less sensitive to illumination changes than pixel intensity, so the calculated disparity is more reliable than that of the intensity based matching method in the case of varying illumination. However, this method O pe n A cc es s D at ab as e w w w .ite ch on lin e. co m